1. Technical Field
This invention is directed toward a system and method for capturing and transmitting meeting content. More particularly, this invention is directed towards a system and method for capturing and/or transmitting the whiteboard content in real-time.
2. Background Art
Meetings constitute a large part of many workers' working time. Making more efficient use of this time spent in meetings translates into a big increase in productivity.
Although whiteboard sessions are frequent for knowledge workers, they are not perfect. The content on the board is hard to archive or share with others who are not present in the session. People are often busy copying the whiteboard content to their notepads when they should spend time sharing and absorbing ideas. Sometimes they put a “Do Not Erase” sign on the whiteboard and hope to come back and deal with it later. In many cases, they forget or the content is accidentally erased by other people. Furthermore, meeting participants who are on a conference call at remote locations are not able to see the whiteboard content as the local participants do. In order to enable this, the meeting sites often must be linked with expensive video conferencing equipment. Such equipment typically includes a pan-tilt-zoom camera which can be controlled by the remote participants. This configuration is still not always satisfactory because the viewing angle, lighting variation, and image resolution are often inadequate. The lack of functionality of effective archiving and indexing of whiteboard contents is also problematic.
Many technologies exist to capture the whiteboard content automatically. One of the earliest, the whiteboard copier, is a special whiteboard with a built-in copier. With a click of a button, the whiteboard content is scanned and printed. Once the whiteboard content is on paper, it can be photocopied, faxed, put away in the file cabinet, or scanned into digital form. Recent technologies all attempt to capture the whiteboard in digital form from the start. They generally fall into two categories—image capture devices and pen tracking devices.
The devices in the first category capture images of the whiteboard directly. NTSC-resolution video cameras are often used because of their low cost. Since they usually do not have enough resolution for a typical conference room size whiteboard, several video frames must be stitched together to create a single whiteboard image. The ZombieBoard system [10], deployed internally at Xerox's Palo Alto Research Center, uses a Pan-Tilt video camera to scan the whiteboard. The Hawkeye system from SmartTech opts for a three-camera array that takes images simultaneously. Another device in this category is the digital still camera. As high resolution digital cameras get cheaper, taking snapshots of the board with a digital camera has become a popular choice. To clean-up the results, people use software to crop the non-whiteboard region and color-balance the images.
There are several disadvantages of the aforementioned image capture devices, however. For example, they capture the whiteboard content one snapshot at a time so users have to make a conscious decision to take a snapshot of the whiteboard. Additionally, there is usually a lag between writing on the board and taking a snapshot. Using these devices in real time teleconferencing scenarios is not very natural or convenient, if possible at all.
Devices in the second category track the location of the pen at high frequency and infer the content of the whiteboard from the history of the pen coordinates. Mimio by Virtual Ink Corp. is one of the best systems in this category. Mimio is an add-on device attached to the side of a conventional whiteboard and uses special adaptors for dry-ink pens and an eraser. The adapted pen emits ultrasonic pulses when pressed against the board. Two receivers at the add-on device use the difference in time-of-arrival of the audio pulses to triangulate the pen coordinates. Since the history of the pen coordinates is captured, the content on the whiteboard can be reconstructed in real time. And because the content is captured in vector form, it can be transmitted and archived with low bandwidth and storage requirements.
Electronic whiteboards also use pen tracking technology. They go one step further by making the whiteboard an interactive device. For example, the SMARTBoard from SmartTech is essentially a computer with a giant touch-sensitive monitor. The user writes on the monitor with a special stylus which is tracked by the computer. The computer renders the strokes on the screen wherever the stylus touches the screen—as if the ink is deposited by the stylus. Because the strokes are computer generated, it can be edited, re-flowed, and animated.
Pen-tracking devices also have several disadvantages, however. Pen-tracking devices require instrumentation either to the pens and erasers or to the surface that they are writing on. For example, Mimio uses special adapters for dry-ink pens, which make them much thicker and harder to press. Electronic whiteboards are not even compatible with the existing whiteboards. They use touch screens as their writing surfaces, which limits their install base due to high cost and small size. If the system is not turned on or the user writes or erases without using the special pens or erasers, the content cannot be recovered by the device. Many people like to use their fingers to correct small mistakes on the whiteboard. This common behavior causes extra strokes to appear on the captured content. Additionally, there is usually minor imprecision in the tracked pen coordinates, which tends to accumulate and cause mis-registrations among the neighboring strokes. Furthermore, pen-tracking devices do not allow multiple users to write on the whiteboard simultaneously. The image capture devices do not have this problem since they work in a What You See Is What You Get (WYSIWYG) manner.
In addition to whiteboard capture devices, much research has been done on the capture, integration, and access of the multimedia experience. People have developed techniques and systems that use handwritten notes, whiteboard content, slides, or manual annotations to index the recorded video and audio for easy access [1,2,4,6,7,8,9,10,11,12,13,14]. Inspired by those systems, a Whiteboard Capture System (WCS) was developed [0]. The goal of that project was to build a whiteboard capture system that combines the benefits of both image capture devices and pen tracking devices. The key design decision that was made was to use an image capture device as input. Without the requirement for special pens and erasers the interaction with the whiteboard is much more natural. Furthermore, since this WCS takes images of the whiteboard directly, there is no misregistration of the pen strokes. As long as the users turn on the system before erasing, the content will be preserved. Additionally, images captured with a camera provide much more contextual information such as who was writing and which topic was discussed (usually by hand pointing).
Although the WCS can readily and accurately capture key images and associated audio, its processing is too complex and time consuming to operate in real-time. Additionally, since a particular digital camera was chosen as the input device for one embodiment of the WCS for its high resolution (4 MP) and the availability of a software development kit, which allows one to control the camera from the PC, complex camera control is not necessary. However, because the camera is connected to the host PC via low bandwidth USB 1.1, the frame rate is limited to 5 second per frame. At such a low frame rate, no attempt could be made to use the WCS as a real time conferencing tool. Therefore, the previous WCS was designed to analyze and browse offline meeting recordings. From the input image sequence, a set of key frames that captures the history of the content on whiteboard and the time stamps associated with each pen strokes was computed. A key frame contains all the visual content before a major erasure. This information can then be used as a visual index to the audio meeting recording. Aside from the inability to operate in real-time, the previous WCS also had to be calibrated every time it was used or installed.
Therefore, what is needed is a system and method for capturing whiteboard content in real time that allows the recording and browsing of the content after the meeting, as well as transmission to remote meeting participants in real time. The system should not require expensive specialized video conferencing equipment and should be easy to set up and use. Additionally, transmission to remote meeting participants should use as little network bandwidth as possible to transfer the whiteboard content making the system suitable for even dial up network connections.